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Assessing the Effectiveness of Public Research Universities 

Using NSF/NCES Data and Data Envelopment Analysis Technique 

Abstract 

This study explores the use of Data Envelopment Analysis (DEA) as a multi-dimensional 
and multi -criteria tool for assessing and benchmarking the performance of public research 
universities through the use of national databases such as the annual institutional surveys 
conducted by the National Science Foundation and the National Center for Education Statistics. 

DEA, as a tool for obtaining multivariate performance indices, has been used extensively 
to study the performance of public sector and service-oriented private organizations. In studying 
the performance of higher education institutions, DEA's mathematical formulation for measuring 
university performance can be verbally stated as: "find a multivariate ratio which (1) 
characterizes each university in terms of its outcomes and resources, and (2) provides an 
ordering, from best to worst, of universities with similar outcomes, resources, and environmental 
constraints." 

In applying the DEA method, we examine the research and instructional outcomes of 
public research I universities and perform an in-depth analysis of the Ohio State University. Our 
analysis progresses from simple three-variable models to a full-blown model with multiple 
variables for the comprehensive assessment and comparison of university performance. The 
results of this study demonstrate how DEA as an "attention focusing" device can be used by 
university decision-makers to assess their strategic positions and to locate key areas for 
improvement. 
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Assessing the Effectiveness of Public Research Universities 1 

Using NSF/NCES Data and Data Envelopment Analysis Technique 



Introduction 

The annual release of US News and World Report's college ranking brings "cries of elation, 
howls of anger, or muted mutterings in the offices of college administrators throughout the land" 
(Sanoff, 1998). Despite the controversies surrounding them, college ranking systems are here to 
stay. In addition to US News, Time, Money, Forbes and various other publications are all eager 
participants of the ranking game. Recently, there has also been talk of designing a national "report 
card" system to grade the performance of higher education institution in all 50 states (Sellingo, 2000) 
and creating a ranking of U.S. research universities and their international peers (Southwick, 2000). 

This popularity of college rankings, while reflecting its appeal as a marketing tool for 
newsmagazines, underscores a significant shift in both public opinion and policy attention on the 
outcomes of higher education. With the cost of higher education continues to escalate at a rate much 
higher than inflation, taxpayers and tuition-paying parents justifiably demand to know whether their 
money is well spent. Sensing the public's greater level of scrutiny, state governments and 
legislatures have begun to link funding for public universities to their “performance.” In Ohio, 
Governor Bob Taft has asked that the Board of Regents issue annual "Report Cards" for the state's 
public colleges. Other states already have such systems in place. For instance, in South Carolina, 
funding for the state's public universities is determined by a balanced scorecard system that 
comprises of 37 performance indicators (SCCHE, 1999). 

Assessing the performance of institutions of higher learning poses unique challenges. The 
primary challenge comes from their diversity. The 1987 Carnegie classification identifies nine 
different types of institutions of higher learning where each group has its own set of goals and 
mission. Across these different types of institutions, there is tremendous diversity in the student 
body, expectations and standards regarding the quality of teaching and level of research 
commitments. Many commonly used performance measures suffer from several weaknesses: they 
have a mechanistic view of higher education; they offer partial and often simplistic views based on 
readily available data, and they are often applied without due regard for institutional differences 
(Gaither, 1995). Some ranking systems rely heavily on input indicators such as ACT or SAT test 

1 This paper is still a work in progress. Please do not cite or quote without the authors' explicit consent. 
The authors would like to express their gratitude to Dr. Anand Desai for his technical assistance and 
helpful comments 
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scores. This reliance may create an unintended and pernicious effect in keeping many perfectly good 
applicants, many of them minorities, from higher education institutions of their choice. 

Being mindful of the need for a more systemic and comprehensive performance assessment 
tool for higher education institutions and the inherent limitations of commonly used methods, we 
pursue the exploration of Data Envelopment Analysis (DEA) technique as an alternative approach in 
assessing the overall performance of higher education institutions. Our objectives are to demonstrate 
how DEA, when coupled with national databases, can complement traditional performance measures 
and to provide an alternative approach for universities to examine their overall performance in 
comparison to newsmagazines' rankings. 

Specifically, we utilize publicly available data that have been collected over time in a 
consistent manner and determine how they might be used in a performance assessment of institutions 
of higher learning. These data are available primarily from the National Science Foundation (NSF) 
and the National Center for Educational Statistics (NCES). We attempt to couple these data 
resources with a powerful analytical tool called Data Envelopment Analysis (DEA) to develop a 
multi-dimensional, multi-criteria method for developing performance measures and compare and 
contrast its findings with other commonly used performance measures. We present in this paper the 
findings about the instructional and research performance of America's public Research I universities 
as defined by the Carnegie Classification System. 



Performance Measurement in Higher Education 

Performance measurement practices in higher education have, by and large, followed 
developments in organizational theory where there are three major approaches to performance 
assessment: input-output ratio analysis, outcome-based assessment, and stakeholder-based evaluation 
(Altschuld and Zheng, 1995). Depending on the core value systems of the institutions and their 
assessment objectives, different approaches have different utilities (Campbell, 1977; Rohrbaugh, 
1983). Each of these approaches is useful and relevant, depending on the degree to which it fits the 
specific needs and situations of the organizations concerned. Typically, a number of performance 
indicators are used to reflect an organization's relative position in utilizing its resources, managing its 
organizational processes, and achieving outcomes. These performance indicators are typically ratio- 
based quantitative measures in ordinal, cardinal or comparative terms (Kells, 1994). Many 
universities compare their performance indicators with those of a group of peer schools in a process 
commonly known as "benchmarking" (Alstete, 1995). 
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Individually, each performance indicator is useful in identifying the position of a university 
relative to its benchmark peers in a specific performance area. However, when multiple performance 
indicators are involved, problems arise. For example, some universities may choose to emphasize in 
certain areas for strategic purposes. Hence, a university may be ranked lower in some areas but 
higher in others. To solve this problem, a weight is often assigned for each performance indicator 
(i.e., the balanced scorecard). Such a weighting scheme is often applied uniformly without due regard 
for institutional differences. Because they often involve subjective judgement and political 
negotiations, weighting schemes are also controversial. Furthermore, ratio-based measures generally 
entail comparisons with the mean or median values in order to determine whether the performance of 
the unit in question is above or below “average”. These comparisons rarely capture the complexity 
inherent in the multidimensional nature of performance evaluations (Desai, 1999). 



DEA and Its Use in Performance Measurement 

Over the last two decades, Data Envelopment Analysis (DEA) has emerged as a truly multi- 
dimensional approach to assessing the overall performance of organizations (Seiford, 1990). While 
some of the earliest applications of DEA were in the area of elementary and secondary education 
(Charnes, Cooper and Rhodes, 1978; Bessent and Bessent, 1982, Desai, 1986; 1992), its application 
to measure the performance of university faculty (Walters, Cornia and Cameron, 1997; Walters, 
Cornia and Chabries, 1998), of university departments (Johnes and Johnes, 1993; Sinuany-Stern, 
Mehrez, Barboy, 1998; Tomkins and Green. 1988), and of institutes of higher learning as a whole 
(Kao, 1994; Sarrico, Hogan, and Athanassopoulos, 1997), is fairly recent. 

In a study of Canadian public universities, McMillian and Datta (1998) use DEA to 
demonstrate how university decision makers can use DEA study results to examine their standing 
relative to their peers and assess the strategic planning options with the information generated. 
McMillian and Datta control the contextual variations by separating sampled Canadian public 
universities into three groups: comprehensive with medical school, comprehensive without medical 
school, and undergraduate only institutions. They use several models with different mixes of input 
and output variables. Their findings suggest that despite their use of different variables, small 
subsets of universities are consistently ranked as “best-practice” performers. This small group of 
universities represents a set of reference points against which other schools may find their relative 
positions in overall performance. 
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In another study, Haksever and Muragish (1998) use DEA to measure the outcomes of MBA 
programs in the United States. Their study focuses on the added values of MBA program. They use 
the average starting salaries and job placement rate of MBA graduates as the primary outcome 
variables. Their assessment takes into account the differences in faculty, financial, and student 
resource available to each school. Their study finds that despite general perceptions, public 
universities provide as much added values to their MBA graduates as their private counterparts. 
Moreover, regardless of public and private sector differences, the top 40 MBA programs vary very 
little in their effectiveness in generating added value. 

Because of its relative newness, studies of organizational effectiveness in higher education 
through the use of DEA technique are mostly exploratory in nature. However, the small number of 
studies that have been done show clear promise that DEA, when coupled with reliable data, can be 
used to map the relative performance of universities with similar missions and contextual constraints. 
Minimally, a DEA study can serve as an attention-focusing device to help higher education decision- 
makers to identify peer groups for comparison or for locating strategies for institutional 
improvements. One nice feature of DEA analysis is that its findings can be used to locate the optimal 
resources allocation strategies for organizations with different mission focuses. Organizations can be 
compared by how they managed their resources or by how they achieve their outcomes. 



Data Envelopment Analysis - A Brief Introduction 

DEA has its origins from economic theory of production and linear programming. Linear 
programming is concerned with the general problem of allocating limited resources among 
competing activities in the best possible way. Hillier and Lieberman in their classic textbook, refer 
to linear programming as being “among the most important scientific advances of the mid-twentieth 
century” (1984, p.29). Chames, Cooper, and Rhodes (1978) are credited with the development of 
DEA based on the principles of linear programming. DEA provides a quantitative measurement tool 
for evaluating the efficiency of public and non-profit organizations, where some inputs and outputs 
cannot be measured in monetary units and therefore may not have a “bottom-line” for measuring 
performance. 

Through DEA analysis, efficiency can be examined in two ways: maximizing the outputs 
given a certain level of inputs or minimizing the inputs given a certain level of outputs. Consider 
monitoring the performance of a university that, for the sake of simplicity of exposition, uses two 
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resources, R1 and R2 to achieve two outcomes, Ol and 02. Using data on these four variables from 
a number of universities, we can obtain figures similar to those shown in Figure 1 . 

In the figure on the left, the axes measure two resources R1 and R2 being used to produce the 
desired outcomes. The data from the different universities would yield a scatter plot. Assuming that 
the three universities, A, B and C represent the minimal combinations of resource utilization they 
identify the best-observed practice. The frontier, RABCR denotes a "best practice frontier." We 
may thus construct performance measures based on the distance from the frontier. Similarly, the 
universities can be compared on the basis of outcomes achieved. Assuming more to be preferable in 
this instance, universities S, T, U, and V would define the best practice frontier in the figure on the 
right. Thus, distance from the frontier OSTUVO could be used to define outcome performance 
measures. As in the case of resource utilization, universities on the frontier (that is, S, T, U and V) 
are deemed to be effective while the other agencies "below" the frontier are deemed to be lagging 
behind in their performance. 




Figure 1: Best Practice Frontiers 



To provide a simplified illustration of DEA, let us use the benchmarking of university 
research performance as an example. In this example, only three variables are used: research 
expenditure, number of faculty, and number of journal publication. From these three variables, we 
come up with two ratio-based measures: number of publication per $1000 research spending and 
number of publication per faculty FTE. The graph below shows how DEA can be used in such a 
simple case to identify best practices and identify performance gaps. 
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The graph on next page illustrates how DEA maps the performance frontier. Points A, B, C 
and D connect to form a curved line that defines the frontier of research outcome. College A 
represents the highest possible publication numbers per $1000 research expenditure whereas College 
D represents the highest possible publication per faculty FTE. Any school that lies on the line 
ABCD is considered effective and a best practice school. Any school that lies under the best practice 
frontier is considered less effective (not the best). A university that is not on the best practice frontier 
needs to look for best-practice schools that have similar resource allocation structures to improve its 
performance. For example, E has the same publication outcome per $1000 investment as C but 
lower publication per faculty input. E also has the same publication numbers per faculty input as B 
but lower publication outcomes per $1000 research expenditure than B. Either B or C can be E's 
best-practice reference. E needs to emulate B's or C's practices in order to improve its research 
effectiveness. 

Publication / $1000 Spent 




Figure 2: Simple Case of Research Productivity 

Of course, this illustration oversimplifies the issues. Performance assessment in higher 
education is far more complex than this example and requires the simultaneous consideration of 
multiple variables that reflect not only the inputs, outputs, but also the process and environmental 
factors. However, DEA models with more than two variables cannot be easily represented 
graphically. In such cases, we need to formulate the problem in mathematical terms. 
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The mathematical formulation of the problem is not restricted to two dimensions and the best 
practice frontiers as well as the performance indices can be readily constructed using multiple 
resources and multiple outcomes. In the context of university performance, the index that is to be 
developed corresponds to the following verbal statement: 

Find a multivariate ratio , which (I) characterizes each university in terms of its outcomes 
and resources and (2) provides an ordering , from best to worst , of universities with 
similar outcomes , resources ; and environmental constraints . 

A multivariate ratio that meets the above description can be expressed, for each of n 
universities, as: 

<■> 

r i 

Where: Y r j - Level of outcome r of university j; 

X[j = Amount of resource i being used by university j; and 
u r j and W[j are weights assigned to the outcomes and the inputs. 

In creating DEA, Chames, Cooper and Rhodes (CCR) (1978) proposed a mathematical 
programming formulation of this problem which simultaneously solves for the weights and provides 
a relative ordering of these programs. Ideally, each university would want to maximize (1). Since 
the weights are to be “objectively” assigned so as to maximize this ratio, there is no upper limit to the 
size of the ratio. Hence, in order to bind the maximum value that any program could obtain, CCR 
proposed an upper bound of 1 . CCR operationalized the above verbal statement of the problem as 
follows: 



Find weights u r j and W[j such that the ratio of virtual outcomes to virtual inputs is 
maximized subject to the constraint that no ratio exceeds unity. 

Thus, we have in (1) the ratio of a virtual outcome to a virtual resource, where the virtual 
outcome is the weighted linear combination of outcomes and the virtual resource is the weighted 
linear combination of resources. Given this characterization of the university's activity, the 
computational issue to be addressed is (a) how should the weights u and w be obtained and 
subsequently, (b) how should the ordering of these programs be achieved. 

Hence the performances score, h k , for the kth university given n universities using m 
resources to result in s outcomes is obtained by solving a fractional program (Desai, 1999). Charnes 
and Cooper (1962) developed a transformation that yields a linear equivalent for this fractional 
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program thereby considerably simplifying the computation of h^. Hence, the computation of these 
performance scores entails solving the following linear program: 



Maximize: 
Such that: 



h k = Yj h Y 
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The performance score for each university is obtained by solving n such linear programs, one 
for each university in the study sample. Computer software for efficiently solving such complex 
mathematical programs is readily available. 



Study Scope, Variables and Models 

For this study, we focus on the assessment of America's public Research I universities as 
defined by the Carnegie Classification System. The restriction of the research scope to only public 
Research I universities is an attempt to control for variations in missions, organizational structures, 
human capital and financial capacity. Even among public Research I universities, we recognize the 
diversities in academic emphases and strategic orientations. We take into account these differences 
by selectively controlling for mission, contextual and functional differences among these institutions. 
For instance, in assessing instructional outcomes, we control for student selectivity, assuming that 
universities with higher admission standards have a higher probability of student success. In 
assessing research outcomes, we control for the availability of medical research centers and hospitals, 
assuming that universities with medical research centers and hospitals have greater advantages in 
obtaining grant funding and in publishing research findings. 

The use of DEA to analyze organizational performance has its unique data requirements. 
One of these requirements is that all resource and outcome variables be specified and measured 
consistently across all measuring units. Failure to include a valid resource or outcome variable could 
result in a bias. Thus, due care must be taken in identifying relevant variables that are valid for all 
universities. To fulfill this requirement, we turn to the databases available from Integrated Higher 
Education Data Systems (IPEDS) of the US Department of Education and WebCaspar from the 
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National Science Foundation. We believe that both agencies have by far the most consistent and 
accurate data resources for this type of analysis. The fact that IPEDS and WebCaspar have multiple 
years of data affords us the opportunity to examine organizational performance not only for a 
particular point in time but also longitudinally over a long span of time. 

Table 1 summarizes the key variables that we use in this study. This list is part of a much 
longer list of variables that we use in a broader examination of the performance of higher education 
institutions in the United States. It should be noted that all variables, except "six-year graduation 
rate" and the two control variables, are three-year averages of data reported for FY 1995 - FY 1997. 
The use of three-year averages instead of data for a particular year is to avoid bias caused by possible 
fluctuations in a university's performance. This is especially true in assessing research outcomes 
whereas an occasional dip or jump in publication or citation numbers may or may not represent the 
normal state. 



Table 1 

Key Variables Used in This Study 





Instructional Variables 


Research Variables 


Inputs or 
Resources 


• Instructional Faculty FTE 

• Student-related Expenditure per 
Student FTE 

Instructional Expenditure 
Student Services Expenditure 
Scholarship/Fellowship 

• Faculty to Student Ratio 


• Total Faculty FTE 

• Research-related Expenditure per 
Faculty FTE 

Direct Research Expenditure 
Academic Support Expenditure 
Plant & Equipment Expenditure 

• Number of publications 


Outputs 


• Weighted Graduation Efficiency 
Score 

Total Student 
Undergraduate 
Graduate / Professional 

• Six-year Graduation Rate 

• Degree completions 


• Number of publications 

• Number of citations 

• Government Grants and Contracts 
received 

• Total Grants and Contracts received 


Control 


• Admission Selectivity 


• Availability of Medical Research 
Centers and Hospitals 



In assessing instructional performance, the following input variables are used to measure 
available resources: (1) total number of Full-time Equivalent (FTE) of instructional faculty (IPEDS - 
Faculty Salary Surveys), (2) expenditures per student FTE on instruction, student services, and 
scholarships and fellowship (the sum of these three expenditure is called student-related 
expenditures), and (3) instructional faculty to student ratio. The output variables include weighted 
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graduation efficiency score (separated for total student, undergraduate, and post-baccalaureate), six- 
year graduation rate (institution self-reported), and degree completions. 

One of the key output variables, Weighted Graduation Efficiency Score (WGE_Score), is 
calculated based on the following formula: 

UG _ deg+ 1 .5^ M _ deg+ 1 .75^ P _ deg+ 2^ Doc _ Deg 
Total _ Student _ Headcount 

Whereas the variables UG_deg, M_deg, P_deg, and Doc_deg represent total numbers of 
baccalaureate, masters, professional, and doctoral degrees, respectively. 

The WGE_score is basically the ratio between number of total degree completions and the 
total number of students in a specific period of time. Logically, it is reasonable to assume that the 
higher the score the more successful a university is able to produce graduates. Universities with low 
attrition rates and high graduation rates are by definition more likely to have higher WGE_score. 
Traditional graduation rate, like the one used by US News and World Report, measures only the 
success of students who enter the university as first-time freshman. The success of transfer students 
or students of subsequent degrees are not considered in the traditional graduation rate equation. A 
key advantage of the WGE_score is its ability to reflect comprehensively a university's overall effort 
in adding values to the society by graduating not only traditional undergraduates but also non- 
traditional students or students of more advanced degrees. This measure to a certain extent 
minimizes the potential bias against universities that have greater percentage of part-time or non- 
traditional students. 

The WGE_score is a weighted score. We assign a unit weight of 1 to baccalaureate degrees. 
We assign higher weights to post-baccalaureate degrees: 1.5 to master degrees, 1.75 to professional 
degrees, and 2.0 to doctoral degrees. These weights are subjectively assigned to reflect 
considerations that the resources devoted to educating various levels of graduates are different and 
that the society places different values on various levels of degrees. These subjective weights are by 
no means uncontestable and may be adjusted when we have more information and time to evaluate 
their usefulness. 

In assessing research performance, the input variables include: (1) total faculty FTE, 
inclusive of instructional, research, and public service faculty; (2) research-related expenditure per 
faculty FTE, comprised of direct research, academic support, and plant and equipment expenditures; 
(3) total number of publications (also an output variable). On the output side, the variables for assess 
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research performance include: (1) total number of publications; (2) total of citations of publications; 
(3) total amount of government grants and contracts generated; and (4) total public and private grants 
and contracts generated. 

Student admission selectivity (percent of applications rejected) and availability of university 
hospitals or medical centers are used as control variables when we assess instructional and research 
performance separately. Student selectivity provides a proxy variable to determine the input quality 
of a university's incoming students. In assessing instructional performance, we divide the sample of 
58 public Research I universities into two groups: those with higher than 40% rejection rate and 
those below or 40% rejection rate. In assessing research performance, we divide the sample 
universities into two groups: universities with and without medical centers or hospitals. 

In this paper, our models for assessing the performance of public Research I universities are 
outcome-focused and assume a constant rate of return for the production function. In other words, 
we assume that the same amount and mixture of resource should generate a similar level of outcomes 
for all the universities in question. For example, a graduate from UC-Berkeley is considered equally 
valuable as a graduate from any other public Research I universities. With these assumptions in 
place, we seek to maximize the outcomes given the levels of resources available to universities. 
Universities are compared based on their ability to maximize their outcomes in relation to their 
available resources. In such an outcome-based evaluation, universities who can maximize their 
outcomes based on the level of resources available to them are considered as "best practice" 
organizations. Other universities who are not ranked as "best practice" organizations will be arrayed 
into different ranks by their efficiency scores. 

In this study, we use a stepwise approach in conducting the DEA study process: we start with 
one output variable and two input variables and progress to full-blown models with multiple input 
and output variables. In the following sections, we will present findings from models from the two 
extremes: simple models with minimum number of variables and more sophisticated models with 
multiple input and output variables. 



Research Finding I: DEA Analysis with Minimum Number of Variables 

In performing DEA studies, the minimum number of variables that are required is three: 
either one input or two output variables or two input and one output variables. We can also specify 
the model option to either maximize the output given the resources available or minimize the 
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resource inputs given the level of output generated. In this study, we choose to focus on the output 
maximizing aspects of the DEA analysis. 

The universities included in this study are all public Research I universities. Three Research 
I universities are excluded due to missing data: Rutgers University at New Brunswick, Temple 
University and University of California at San Francisco. Most of the data elements come from 
IPEDS surveys and NSF WebCaspar. We also use data from Institute for Scientific Information and 
the US News and World Report. Most of the variables reflect three-year average or three-year total 
from fiscal year 1995 to 1997. The decision to use three-year data rather than one single year is to 
recognize the fact that institutional performance may fluctuate from year to year and a longer time 
span will provide a more realistic reflection of an institution's normal state of affairs. 

The selected variables are divided into two groups: research outcome variables and 
instruction outcome variables. In Table 2 and Table 3, we present findings from several basic DEA 
models with samples limited to schools that share Ohio State University's basic characteristics. For 
example, for research performance, Table 2 covers only public Research I universities with medical 
research centers or hospitals. For instructional outcome evaluation, Table 3 includes only public 
Research I universities with less selective admission standards (admission rejection rate lower than 
30% in 1996). In the following discussion, we will discuss findings presented in Table 2 and 3 with 
graphical illustrations on how the information from even the basic models can help us understand 
research and instructional performance for Ohio State University. 
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Table 2: Research Performance -- Basic Models 



Public Research Univerisities 
With Medical Ctr/ Hospitals 


Model 1 
Publications 


Model 2 
Citations 


Model 3 

Grants & Contracts 


Score 


Rank 


Score 


Rank 


Score 


Rank 


COLORADO STATE 


57.3 


33 


44.1 


33 


49.8 


..-7; 29 


IOWA STATE 


62.4 


29 


44.3 


32 


44.7 


35 


LOUISIANA ST UNIV 


74.0 


" 21 


43.5 


34 


29.1 


41 


MICHIGAN STATE 


62.9 


27 


59.1 


18 


55.3 


25 


NORTH CAROLINA STATE 


62;6 


• 28 


40.6 


37 


r ‘ ’ 7 50.0 


1 vlllT^ 


OHIO STATE UNIVERSITY 


85.9 


9 


52.3 


25 


72.6 


14 


OREGON STATE UNIV . , 


. 43.6 


71 .38 


39.5 


, 77 39 


2748-9 


177. ■ 77.731 


PURDUE UNIVERSITY 


82.7 


10 


46.0 


31 


55.8 


23 


SUNY AT BUFFALO 


69,9 


: Tl 22 


55 5 


’7 ; : 23 


.2 177Z#-2 


ll' : 7l32 


SUNY AT STONY BROOK 


92.4 


8 


73.7 


9 


57.5 


21 


TEXAS A & M UNIVERSITY £ •. / 


v \ 82.6 


.... - ; 1 


12 Zll 2 - 9 


- 22 ■ .7736 


27 12763.1 


\:7. 2£;)47 


UNIVERSITY OF ALABAMA 


67.4 


25 


86.9 


6 


74.0 


'13 


UNIVERSITY OF ARIZONA 


78.9 


J9 


59.5 


_ _ .’112 17 


2. 7.1:748 


11 .71132 


UNIVERSITY OF CA-DAVIS 


81.8 


12 


56.4 


22 


577 


20 


UNIVERSITY OF CA-IRVINE 


69.7 


V. 23 


, —— 71 8 


7 Jo 


: 1 7 53*4 


71. 271726 


UNIVERSITY OF CA-LOS ANGELES 


100.0 


1 


87.2 


5 


95.9 


js 


UNIVERSITY OFCA-SAN DIEGO : 2 


^£2 1 98 9 


212121s 






1:T9SS 


7. ■IITp 


UNIVERSITY OF CINCINNATI 


62.0 


30 


63.8 


15 


42.6 


37 


UNIVERSITY OF CONNECTICUT ' 


71' 1 7 80.4 


. ““je 


• . .27.50,1 


'Sjjl J3 


. ,7: 32-5 


7171^40 


UNIVERSITY OF FLORIDA 


81.1 


15 


49.0 


29 


62.4 


~ 18 


UNIVERSITY OF GEORGIA '" I . 


71 513 


7 36 


1 .39.6 


. 33 


' 753.0 


71 7:1727 


UNIVERSITY OF HAWAII 


40.9 


40 


51.1 


27 


44.0 


36 


UNIVERSITY OF IL AT CHICAGO 


58.9 


31 


52.3 


• - " , : 2 6 


'.7 777 . 49.4 


17 ' ’ 1 36 


UNIVERSITY OF IL AT URBANA 


79.3 


18 


53.9 


24 


75.5 


11 


UNIVERSITY OF IOWA 


80.3 


71 7 Ui 7 


77.’ % 68.1 


.112 


12 72765.8 


171 111 6 


UNIVERSITY OF KENTUCKY 


54.8 


34 


48.5 


lo 


^ 45.6 


34 


UNIVERSITY OF MICHIGAN 


71 1 loo.o 


1'* 11.1 J 


7* V 2 78.9 




77 1799.4 


. 21" '3' 23 


UNIVERSITY OF MINNESOTA 


100.0 


1 


69.3 


Ti 


100.0 


" 1 


UNIVERSITY OF MO-COLUMBIA 


81.2 


' ^ 14 


43.2 


36 


l” *"1236.4 


7 11739 


UNIVERSITY OF NEW MEXICO 


43.6 


37 


57.2 


20 


76.2 


10 


UNIVERSITY OF NORTH CAROLINA 


81:7 


13 


87.3 




17 77 7 98.6 


■ *7 7 717 4 


UNIVERSITY OF PITTSBURGH 


100.0 


1 


96.4 


3 


83.2 


8 


UNIVERSITY OF TENN-KNOXVILLE 


‘71, . • 75.8 


717 22 20 


7L,„, 77 56.6 


2 \:211 2i 


77 7:739.3 


21172138 


UNIVERSITY OF UTAH 


66.7 


~ 26 


81.9 


7 


66.0 


*15 


UNIVERSITY OF VIRGINIA _ .."2 . 


1; 89-6 


77 -124 


'2 . 7. 66.5 


77171*74 


27 2 77.2 


' 11- -9 


UNIVERSITY OF WASHINGTON 


100.0 


1 


100.0 


i 


100.0 


1 


UNIVERSITY OF WISCONSIN 2 


95.3 


r l ■ 


12 . 111 © 


7 " |6 


•'17 7i?3.o 


77711 1 7 


VIRGINIA COMMONWEALTH UN 


42.7 


~ 39 


67.1 


13 


55.7 


24 


VIRGINIA POLYTECHNIC INST ' 


’• 52.3 


j; 35 


7 " 711© 


17.740 


7 " ; . * 2 5*7.5 


2 7 H 22 


WAYNE STATE UNIVERSITY • 


58.5 


32 


58.6 


19 


.59.0 


19 


WEST VIRGINIA UNIVERSITY 


30.2 


41 


31.3 


41 


47.7 


33 



Notes: 

For all models: Inputs = Total Faculty FTE, Research-related Expenditure per FTE 
Model 1: Output = Total Number of Journal Publications 1995-97. 

Model 2: Output = Total Number of Journal Citations, 1995-97. 

Model 3: Output = Total amount of Research Grants and Contracts generated 1 995-97. 
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Table 3: Instructional Performance - Basic Models 



Public Research Universities with 
Admission Rejection Rate < .30 


Modeil 

WGE__Score 


Model 2 

Weighted Total Degrees 


Score 


Rank 


Score 


Rank 


ARIZONA STATE 


87.4 


6 


93.7 


.4 


COLORADO STATE 


75.8 


19 


66.8 


24 


FLORIDA STATE 


100.0 


1 


100.0 


1 ..,-7 7 ; i 


GEORGIA INSTITUTE OF TECH 


97.7 


3 


68.6 


23 


INDIANA UNIVERSITY ~ 7 7^ : ‘ ; ’ : 


' V *“70.1 


— * y . 22 


. V 1 • 83.0 




IOWA STATE 


73.0 


20 


66.6 


25 


LOUISIANA ST UNIV • ' 7 - 


' - . ' c 79.1 


" 7; - ■ ’ ll 


: .. 7’-? 7 , . 155.8 




MICHIGAN STATE 


63.0 


29 


73.7 


18 


OHIO STATE UNIVERSITY 


c . 55.7 


3'::D r ,. 34 


: .785:7 




OREGON STATE UNIV 


100.0 


1 


77.1 


15 


PURDUE UNIVERSITY ” '■ v-r ■■ 


• ' ’* 63.0 


; ; ^T" : K';28 






SUNY AT BUFFALO 


83.4 


9 


86.0 


9 


TEXAS A & M UNIVERSITY 


; "58.5 


31 


: . v . .. . 86 3 


a-‘ ? " 


THE UNIVERSITY OF TX AUSTIN 


79.8 


10 


100.0 


1 


UNIVERSITY ^0F'/5LAB^*' ;T'~' ; " : 




■ < c - ; s 


1&4 


. r , 


UNIVERSITY OF ARIZONA 


63.5 


26 


72.7 


20 


UNI VERSITY OF Cl NQNNAJI £77 . TL 




; 7, 






UNIVERSITY OF CO BOULDER 


71.0 


21 


72.6 


21 


UNIVERSITY OF CONNECTICUT 723 


;A ? 78”Q 


15 


7 ZW 




UNIVERSITY OF IL AT URBANA 


76.1 


18 


87.8 


7 


UNIVERSITY OF IOWA 


-•7 -V. ^ “qq o 


; : V. 24 


■'‘78.9 


4 


UNIVERSITY OF KENTUCKY 


62.0 


30 


544 


34 


UNIVERSITY OF MASS-AMHERST 7. 


r “ v *'t/7 -'63.2 


27 


_ ,3 : <: L.7, • 58*2 


' o T 31 


UNIVERSITY OF MINNESOTA 


~ 57.2 


33 


91.5 


5 


UNIVERSITY OF MO-COLUMBIA 




’ 23 


;;:76J5 


17.."*”;' ?W'§ 


UNIVERSITY OF NEBRASKA 


78.1 


13 


57 .2 


32 


UNIVERSITY OF NEW MEXICO . ; 






y jQC -7-17 




UNIVERSITY OF TENN-KNOXVILLE 


78.1 


14 


65.3 


28 


UNIVERSITY OF UTAH 


76.3 


7 17 


‘ 75.5 


. . . ' 7. '&F 


UNIVERSITY OF WISCONSIN 


64.3 


25 


89.7 


6 


UTAH STATE UNIVERSITY _ 


■ V 3. .3 - 87.2 


• .* 77 7 


• 7:7: 7 Z.59.1 




VIRGINIA COMMONWEALTH UN 


78.3 


12 


79.6 


12 


WAYNE STATE UNIVERSITY .7. 


■ X,-£i93.7 


4 






WEST VIRGINIA UNIVERSITY ~ 


83.8 


8 


79.1 


13 



Notes: 

For all models: Inputs = Instructional Faculty FTE, Student- related Expenditure per Student FTE 
Model 1 : Output = Weighted Graduation Efficiency Score (WGE_Score) for Total Student Population 
Model 2: Output = Weighted total number of degree completions between 1995 - 1997. 
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In Table 2, findings from three basic models are presented. As Figure 3 illustrates, each of 
the models has total faculty FTE and research-related expenditures per faculty FTE as the resource 
variables. The outcome variable for each model is different: total number of journal publications, 
total number of publication citations, and total research grants and contracts received are outcome 
variables for models 1, 2, and 3 respectively. 



Figure 3: Basic Research Performance Models 




Figure 4: OSU vs. U of Washington when Citation is the Outcome 




The Ohio State University (OSU) achieves a high score when publication number is the 
outcome and a low score when citation is concerned. OSU is above the average in generating 
research grants and contracts. The best practice school that OSU needs to emulate is University of 
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Washington who has the highest score in all three models. For example, in Model 3, while 
University of Washington has about 20-30% more resources than OSU, its research impact in the 
form of publication citations is 175% greater. 

With regard to instructional performance, two basic models are used (see Figure 5). The 
input variables for both models are: total instructional faculty FTE and total student-related 
expenditure per student FTE. The outcome variable for Model 1 is the WGE_score (discussed in 
preceding section) and the weighted total degree completions. 

Figure 5: Basic Models of Instructional Performance 




As Table 3 indicates, OSU’s performance in instructional outcomes varies greatly. Among 
public research universities who are relatively less selective in admission standards, OSU is ranked 
quite high in turning out degree recipients (Model 2). However, when the WGE_score is considered 
in Model 1, OSU slips to the bottom of the list. As explained earlier in this paper, WGE_score 
measures the overall value-added effort of the university in producing college graduates. In essence, 
WGE_score indicates the ratio between total degree completions and the total enrolled student 
population in a specified time period. The higher the WGE_score, the more likely a university is 
able to promote student success through better process management and higher retention rates. 

The fact that Ohio State is somewhat efficient in turning out large number of degree 
recipients but lags far behind other schools in overall value-added efforts indicates a strategic trade- 
off. Being the second largest campus in the United States, the huge student base at OSU certainly 
provides an advantage of a critical mass in producing degree recipients. However, despite the 
faculty and financial resources available, the university is not effective in bringing the success rate of 
its student population to a level that is supposed to be. Interestingly, two of OSU’s internal 
benchmark schools, University of Minnesota and University of Wisconsin, both have the same 
problem. Without even getting into the more sophisticated models, it is obvious that these state 
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flagship type schools have ample room to improve the success rate even when they are able to move 
large number of student to graduation. 



Research Finding II: Integrative Performance Analysis with Multiple Variables 

Moving from the basic models to more integrative models which are what DEA 
analysis is best suited, more variables are added. Once again, we separate the research 
and instructional performance variables into two groups and run separate analysis. For 
the purpose of examining the strategic strengths of all public Research I universities, the 
integrative models do not control for variations in student selectivity and the 
availability of medical centers or hospitals. As Figure 5 indicates, each model has 
multiple resources and outcome variables. 



Figure 5: Integrative Models for Evaluating Performance 
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In Table 4, the overall research and instructional performance of all public 
research I universities is summarized. Based on the DEA scores, we make some initial 
assessment on how to understand each university's positions relative to other public 
research I universities in terms of research and instructional outcomes. 
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Table 4: Overall Research and Instructional Performance 



•••;/•> ; 'A, t 

-• •• ’ ■% fet - 

Universities: • v.^ 4\, 


Research -Performance Instructional* Performance 


DEA Score • •,,%‘ : R'0hk- >r DEA Score 


ARIZONA STATE ; r 


. '34..-. '65.7 v.; u>.. 


COLORADO STATE 


58.3 48 100.0 1 


FLORIDA STATE, LI ' , : ISS2 
GEORGIA INSTITUTE OF TECH 


. -3L.: . 

100.0 1 100.0 1 


INDIAN A;yNlVERSI® . 
IOWA STATE 
LOUISIANA SfUNIV F ' ' 


••• • ' r^fr'lop 

63.3 ‘ 44 "* ~89.6 *' 36 


MICHIGAN STATE 


71.1 34 91.6 35 


NEW MEXICO STATE ‘ 
NORTH CAROLINA STATE™ 


it' wwww^w% 

64.9 41 100.0 1 


OH'rOSTATEONIViRSITXF^^I^ 

OREGON STATE UNIV 


52.9 52 100.0 1 


PENNSYLVANIA STATE". 
PURDUE UNIVERSITY 
SUN Y AT BUFF ALO T = ‘”' 


~ 823 " 20 ~~ ^ '^ 100.0 " 


: “ ■ yr , :=^p|: *;.-“a 5 § 2'.9 l 32 


SUNY AT STONY BROOK 
fixASA®©UNra®RsifY’F : S^fi;" 


92.2 14 92.3 33 


" v . ;• , 95,3 *, ?' 'T 3 . 100.0 


THE UNIVERSITY OF TX AUSTIN 


91.9 15 94.2 31 


UNIVERSITY OF ARIZONA 


81 5 22 86.4 37 


UNIVERSITY OF CA-DAVIS 


76.6 30 71.1 49 


UNIVERSITY OEC^RVINE7rF r :,n' f 




UNIVERSITY OF CA-LOS ANGELES 
UNi.VERSrtY OF CA-SAN DIEGO' 


100.0 1 60.2 56 


UNIVERSITY OF CA-SANTA BARBAF 


76.9 29 100.0 1 


UNIVERSITY OFOiNCrNNAILltelfci'' 


"• W, 'W*%. .®*ESL!?' Ms$ WWW^WMSS 


UNIVERSITY OF CO BOULDER 


100.0 1 79.7 41 


U NTV ERSIT YOFOONN ECTICUT' eg- 


“■ «FSoi m 


U NI V E RSITY OF FLORIDA 


78.7 27 75.5 47 


UNIVERSITY OF GEORGIA Vy-\.'*~ 


£t' : 'rU3S1512Mi WZ'&MiS.Sl g?alil. 


UNIVERSITY OF HAWAII 

universityofTl at.chicagoF 


48.5 55 100.0 i 


UNIVERSITY OF IL AT URBAN A 


79.6 26 95.1 27 


university ofioiva r 




UNIVERSITY OF KANSAS 


56.8 50 94.9 28 


UNIVERSITY 6f'KEN jUCK^L^V ; 


K .-.v >; S *' ''It? Jit n 75 . 6 


UNIVERSITY"' OF MARYLAND 


64.4 43 81.7 40 


UNIVERSITY OF.MASS-AMHERST? .. 




UNIVERSITY OF MICHIGAN 


100.0 1 62.3 55 


UNIVERSitY OF MINNESOTA. 


♦?*.;* • ^.. 100,0 • z-'Js* '•ffidsn*?* 


uni ve rsity ofm”c>colu M BI A~ " 


85.1 17 72.7 48 


UNIVERSITY OF WbrMIOVTFI?".^’' 


^TT? : 5^t5!1aS'F " 95.1 


UNIVERSITY OF NEW MEXICO 


83.2 18 78.3 42 


UNIVERSlf Fop NORTH CAROLINA 


i ' ~ TI3SIEIM3- 


UNIVERSITY OF PITTSBURGH 


‘ *100.0 " 1 ’ ~ 67.8 " 50 


UNIVERSitY OF TENN-KNOXVILLE 


:./ ; ’ 7 5 ; '^i^* : Vi->67:6 . c'‘* 37 ' At 100.0 ’ 


UNIVERSITY OF UTAH 


70.9 36 94.3 30 


UNIVERSITY OF VIRGINIA, ^ ,t 

UNIVERSITY OF WASHINGTON 
UNIVERSITY OF WTSCONSIN' 

UTAH STATE UNIVERSITY 
VIRGINIA ^OMWnWEALTFFUW? 


: • 7/ ;. ..•• 79.7 : >25 . -y A, . TOO.O /.;■;/ >4 

' 100.0 ' 1 94.5 29 

58.7 * " ^"47 ’ • ~ — 1 22 


VIRGINIA POLYTECHNIC INST 
WAYNE STATE UNIVERSITY" ."LF” 


’ * 58.9 

7 ■ .: i :£'Iu ? : ;v3kl 


WEST VIRGINIA UNIVERSITY 


51.8 54 91.8 34 
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Strategic Positioning of Universities 

The use of the DEA technique in this study has provided descriptive information on the 
research and instructional efficiency of public Research 1 universities. Though this itself is useful 
information on university performance, the results provided by the DEA analysis can also be used to 
identify the strategic capability of each university. 

In the strategic management literature, superior organization performance is often posited to 
be the result of superior use of resources. In the resource-based theory of organizations (Barney, 
1992) a firm’s strategic use of resources is the basis for competitive advantage. In the resource-based 
view, organizations vary in their ability to effectively leverage common resources and vary in their 
access to unique resources. The extent to which firms or organizations can utilize resources often 
affects their market position and the probability of long term survival. 

From the resource-based research it is apparent that not all organizations have the same 
distribution of resources or the ability to leverage all resources equally well. Thus a common 
approach is to perform at a level which provides competitive parity on many resources, but to 
identify at least one resource or capability in which the organization excels and build the 
organizational strategy from that base. Generally, strategy research has found that maximizing 
efficiency and effectiveness in one resource area is enough of a challenge for most firms. Firms that 
can excel in more than one resource area create multiple forms of competitive advantage and often 
exhibit superior performance. 

Analysis of the extent to which universities excel in the use of their resources can provide 
information about their source of strategic advantage. Identifying a strategic position from empirical 
analysis can inform both policy making and resource allocation decisions within a university. In the 
DEA analysis, results identified the relative position of universities on two resource dimensions, 
research performance and instructional outcomes. 

The two dimensions were distributed into three categories: high, threshold, and low. A high 
score indicated that the efficiency on the dimension was a source of competitive advantage. If a 
school scored 85 or higher on the DEA resource analysis or greater than 90 on the instructional 
analysis it was assigned to the “high” category. If a university scored between 70 and 84 (inclusive) 
on the research dimension and between 80 and 89 (inclusive) on the instructional dimension it was 
assigned to the “parity” category. This indicated that while not a source of competitive advantage, 
the university was operating at an acceptable, but not competitive, level of performance on that 
dimension. If a university received a score below 70 on the research dimension or below 80 on the 
instructional dimension, it was assigned to the “low” category. This score indicates that the 
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university is operating at a low level of efficiency on that dimension. This low level of efficiency 
may or may not be purposeful. Using these distinctions each university was assigned to a strategic 
position based on the combination of research and instructional DEA scores. 

Though subjectively assigned, this strategic typology can be illuminating. The resource- 
based theory suggests that universities that achieve excellence on multiple dimensions will be 
superior performers. Superior performance can also be identified by universities that excel on at 
least one dimension and achieve parity on others. Complete parity is to have no source of excellence 
thus the university is simply “treading water”. What is worse is to utilize resources more 
inefficiently than peers and competitors. This suggests that the university is at a competitive 
disadvantage and will have difficulty in the long run. 

Examining the two dimensions together can also provide insight on the set of tradeoffs being 
made by the university. If resource replenishment is difficult, university administration, either 
explicitly or implicitly, may make choices to support one strategic capability at the expense of 
another. If a university can maintain a threshold level of performance, that suggests the tradeoffs 
between the two performance dimensions have achieved some level of balance. There will be a 
strategic focus, but not at the expense of a parity position. If a university achieves a high efficiency 
on one position and a notable inefficiency on another, this suggests that the university’s strategic 
focus is achieved by purposely cannibalizing internal resources and accepting some level of 
inefficiency within the university on one of the two dimensions. Thus the DEA analysis may show 
the source of advantage for universities and provide some insight about the internal tradeoffs being 
made. 

In Table 5, we show how the 56 Research 1 universities are distributed across the strategic 
typology. The most enviable position is that of “Powerhouse”. The seven universities in that 
dimension are achieving efficiency on both the instructional and the research dimensions. This 
strategic type accounted for approximately 13% of the total sample. The largest category of 
Research I universities occurred in the "Teaching Cannibal" category. Twenty-nine percent of the 
sample was assigned to this category. In the DEA analysis, these schools were characterized by low 
research efficiency with extremely high instructional efficiency. 

Instructional efficiency though means simply that these institutions are producing a large 
number of degrees relative to their resource base. It says nothing about the student experience or 
quality of instruction. This caveat must also be applied to the "Research Cannibal" category (9 
percent of the sample). The schools in this category were of two sub-types. One grouping represents 
highly research intensive universities. They may be cannibalizing their instructional mission by 



actually admitting fewer students and conferring fewer degrees than what is indicated by their size 
and resource base. The other "Research Cannibal" subgroup, though, may more likely be 
characterized by schools that may be trying to build research reputations by reallocating resources 
from the instructional side. In both cases low instructional efficiency would be the result in the DEA 
model. 

The other notable category is "Balanced Teaching". Twenty-two percent of the sample was 
assigned there. These schools are producing graduates at a very efficient rate, but are able to allocate 
enough resources to research to maintain a parity position. Within this category may also be sub- 
groups based on the level of investment in the instructional mission of the institution. 

Strategic Position and Institutional Effectiveness 

To assess the relative effectiveness of the strategic typology, additional more traditional 
measures of effectiveness were overlaid on the efficiency based strategic types. Though the cell 
sizes are small, the trends in the data are quite interesting. Table 6 provides a summary of the cell 
means for student quality data and academic reputation data. The implicit hypothesis is that there 
should be a relationship between the institutional efficiency (as calculated in the DEA analysis) and 
institutional effectiveness (as described by the U.S. News academic reputation score and the percent 
of the freshman class who were in the top 10% of their high school cohort.) Though cell sizes are 
small, we examine the correlation of these data by strategic type. 

The results of this data analysis show some amount of consistency between the efficiency and 
effectiveness measures. Table 6 shows that the Powerhouse type on average has the largest 
proportion of high quality students. The reputation scores of the cells with high research productivity 
are substantially higher than the average scores in the other cells. The overall correlation between 
USN academic reputation score and the DEA research efficiency score is .68. The relationship 
between instructional efficiency and the ability to attract high quality students is not as strong. The 
correlation was only .12 for the overall sample. We conclude that there are additional elements that 
may be necessary to develop a complete picture of the link between instructional efficiency and the 
ability to attract students. 

Though cell sizes are small, cell level correlation analysis was done as well. The correlation 
of the academic reputation and research efficiency differed widely by cell. Interestingly, in the 
"Balanced Teaching" cell, the correlation between academic reputation and research efficiency was 
very low, though the correlation between student quality and instructional efficiency was much 
higher. This result was completely contrary to the result for the overall sample. The instructional 
result was the same for each of the cells associated with high instructional efficiency. In the 



"Balanced Research" and "Research Cannibal" cells, the research correlation was as expected, 
however the instructional correlation showed an inverse relationship between efficiency score and 
student quality. 

In the "Stuck in Parity" cell, correlations on both dimensions were negative; particularly 
concerning research. Given the list of schools in the cell, this suggests that academic reputation has 
not caught up with the research performance of the faculty. Analysis of the data in the two 
"Treading Water" cells suggests that a teaching parity may be more useful than research parity for 
inefficient institutions. The "teaching" institutions were at least able to leverage their instructional 
parity into a reasonable number of quality students. The "research" institutions, though, were not 
able to leverage their research parity into gains in academic reputation. Only the "Teaching 
Cannibal" institutions have a lower average academic reputation. 
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Table 5: Balancing Research & Teaching - Public Research I Universities 



Instructional 

Performance 


Research Performance 


High 


Parity 


Low 


High 


Georgia Tech University 
SUNY at Stony brook 
Texas A&M University 
UC - San Diego 
Univ. of North Carolina 
Texas at Austin 
University of Washington 


Louisiana State University 
Michigan State University 
Penn State University 
Purdue University 
SUNY Buffalo 
University of Alabama 
UC Irvine 
UC Santa Barbara 
University of Connecticut 
U of Illinois - Urbana 
University of Utah 
University of Virginia 


Arizona State Univ. 
Colorado State Univ. 
Florida State University 
New Mexico State 
North Carolina State 
Oregon State University 
University of Georgia 
University of Hawaii 
University of Kansas 
University of Nebraska 
U Tennessee - Knoxville 
Utah State University 
Virginia Commonwealth 
Virginia Polytechnic 
Wayne State Univ. 

West Virginia Univ. 


Parity 


Indiana University 
Ohio State University 
UC - Berkeley 
U of Colorado - Boulder 
University of Wisconsin 


University of Arizona 
University of Florida 
University of Iowa 
University of New Mexico 


Iowa State University 
University of Kentucky 
University of Maryland 


Low 


UC - Los Angeles 
University of Michigan 
University of Minnesota 
U of Missouri - Columbia 
University of Pittsburgh 


UC - Davis 

University of Cincinnati 
U of Illinois - Chicago 
U Mass - Amherst 
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Table 6: Categorization and Strategic Choices 



Instructional 


Research Performance 


Performance 


High 


Parity 


Low 




Powerhouse Universities 


Balanced Teaching 




Teaching Cannibal 




High 


Top 10% 43.8 




Top 10% 


33.4 


Top 10% 29.5 






Top 25% 81.2 




Top 25% 


64.7 


Top 25% 59.4 






Reputation Score 


3.8 


Reputation Score 


3.4 


Reputation Score 3.01 




Research Score 


97 


Research Score 


75.8 


Research Score 


58.2 




Instructional Score 


97 


Instructional Score 


97.7 


Instructional Score 


98.3 




Research/Reputation 




Research/Reputation 




Research/Reputation 






Correlation 


0.54 


Correlation 


0.14 


Correlation 


0.30 




Instructional/Top 10% 




Instructional/Top 1 0% 
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Top 10% 
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Reputation Score 


3.4 
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N=5, 9% of the sample 
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Correlation 


0.01 
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Correlation 
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N=5, 9% of the sample 




N=4, 7% of the sample 
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Discussions of Study Findings and Policy Implications 

The strategic types identified by the DEA analysis can provide useful information to 
institutions interested in pursuing goals associated with instructional and research excellence. 
Generally, performance of universities is evaluated one dimension at a time, even though most 
academic leaders recognize the multiple missions of most universities. Public Research 1 
universities are particularly likely to experience the schizophrenic tension of trying to excel on 
multiple dimensions. This is especially difficult in state environments characterized by limited 
resource expenditures for higher education. 

The value of the DEA method is that it allows academic leadership to more clearly and 
explicitly articulate the resource and productivity requirements associated with institution 
improvement. While many universities strive to be "in the top 20", few leaders have a clear sense, at 
an operational level, of what that entails or exactly how the institution may achieve that goal. 

In addition to helping understand the demands of such an institutional aspiration, the strategic 
positioning that is observable through application of the DEA results also allows institutions to more 
clearly plan the directionality as well as the magnitude of the change necessary. For example, if an 
institution is in the "Treading Water - Teaching" category there are multiple paths to improvement. 
Instruction is at parity and research is below parity. The question is which should be addressed first? 
The detail of the DEA analysis can provide specific information on the resource commitment 
necessary to move the university in one direction of another. One goal may be to eventually become 
either balanced in teaching or in research. DEA can assist in determining the best path through the 
strategic types to achieve that goal. For the "Treading Water - Teaching" cell, the institution has the 
choice of improving efficiency in either instruction or research. Improvement in research will lead 
them to "Stuck in Parity" while an improvement in instructional efficiency may lead them to become 
a "Teaching Cannibal". Neither of these positions are idea, but DEA may help determine which of 
these is the most feasible first step. 

In an environment of higher performance expectations and shrinking levels of financial 
support efficient utilization of resources is becoming a critical part of the strategic assessment of 
many universities. While excellence is desired by both internal and external stakeholders, the path to 
excellence is often difficult to find. Though many academicians and trustees can see the end result, 
clear articulation of the trade-offs and resources required is often lacking. DEA can provide 
clarification and allow articulation of trade-offs to a degree that has yet to be used in most institutions 
of higher education. With this type of analysis, real improvements in resource utilization and 
university performance may be within reach. 
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